Automation Testing Flask Application with Playwright & PyTest examples

Let’s explore how to start implementing end-to-end testing for a web application using Python (the Pytest framework) and Playwright — an open-source tool from Microsoft. We will learn how to setup Playwright, add an E2E test for code coverage in an existing project hosted on GitHub.

Before diving into the practical part, let’s explore the basics of Flask ⬇️

What Flask is and how it works?

Flask is a minimalist web framework written in Python. It includes functionality, allowing developers to extend its capabilities depending on project needs. Flask does not enforce the use of a specific database or parser by default — everything can be configured, making the framework flexible and convenient for various tasks.

Why Coding in Flask?

Key advantages of using Flask in web development:

• Simplicity and minimalism. Flask is a microframework that includes only the essential components needed to build web applications.
• Flexibility and Extensibility. Unlike larger frameworks like Django, it does not impose strict structures or dependencies. It supports integration with third-party libraries and extensions, making it easy to add authentication and work with database tables.
• Built-in development server and debugging tools. It includes tools for convenient application testing and debugging. The interactive debugger helps quickly identify and fix errors.
• Active community. Flask has a large developer community, ensuring access to extensive documentation and technical support.
• Scalability. Despite its lightweight nature, Flask is well-suited for both small and large-scale projects.
• RESTful API Development. Flask is widely used for building RESTful APIs, providing a simple and efficient way to manage HTTP requests.

Testing the functionality of Flask applications is a crucial step in development. Proper testing helps identify potential errors, improve code quality, and ensure system stability.

Key differences: Unit VS E2E testing for Flask application

Testing the functionality of Flask applications is a crucial step in development. Proper testing helps identify potential errors, not only related to code quality and system stability, but it also has an impact on user experience.

In Flask applications, conducting unit tests is used for verifying individual functions and routes, while the purpose of end-to-end tests is to validate the entire application flow, ensuring that all integrated components work correctly from start to finish. E2E tests are effective in simulating real user scenarios.

AQA teams commonly use these automated testing tools for Python, such as PyTest or Playwright frameworks, allowing them to detect errors promptly before the product launch.

In this tutorial, we stay focused on testing rather than API development, we will concentrate exclusively on testing the application. The application development itself, you can learn with the video Flask Tutorial for Beginners of How to build a simple meme Python website by NetworkChuck

We are considering a simple App built with Flask that retrieves data via a public API, which is the content base of the project. Based on it, we will demonstrate how to write three types of tests, API, Unit and E2E, for a Flask application. The sample of this App you can download or clone by following the link.

Once the repository is downloaded, make sure — your Python version is 3.5 or later.

Next, you can install the necessary packages by running the following command:

pip install --upgrade pip

To run this Flask application, execute the following command:

flask run

After that, you can see a message with localhost port where your App is running 👀

Now, when we announce the basics of Flask, it’s time to get acquainted with test framework examples and set it up for our Flask app.

First Pytest test for Flask

Pytest is a convenient Python testing framework that helps developers create higher-quality systems and confidently release them into production. With Pytest, it is easy to write both small and scalable test cases. Below you can find an example of a basic test structure using Pytest.

Pytest Project Prerequisites

First, install Pytest by executing the following command in your project’s terminal:

pip install pytest

Note: If you are working in a virtual environment, you need to activate it before installation. Using a virtual environment is the optimal solution for isolating different projects and their dependencies in Python. To know more, you can use Python Doc virtual env library

After completing the installation, connect the Pytest module. To do this, add the following line to any test file:

import pytest

Store your tests in the tests directory (folder) located in the root of the project. Pytest recommends naming test files in the format test_*.py or with the suffix **_test.py Using these standards simplifies automatic test discovery and helps avoid confusion during execution. In the screenshot below, you can see two files of tests.

In this example of the first Pytest test, we check the response from the API’s default endpoint (/). The API endpoints’ response should be 200 OK. This test fragment is used to check whether we can retrieve a meme. It ensures that the homepage loads successfully and contains the expected content.

But first, to complete the installation, connect the Pytest request module:

pip install pytest requests

Execute the test using pytest:

pytest ./tests

You can also run a specific test file by explicitly specifying its name after the Pytest command:

pytest test_app.py

After executing these commands, Pytest will automatically find all tests in the root directory or the specified file of a standard naming pattern, so you don’t need to specify file or directory names manually.

To provide further verification, we can include a content assertion validation to see if the response matches our expectations, if the client response contains the <h1>title phrase ‘Just Chill and enjoy some memes!’ and is a meme present in the <img> and tag is not empty. Therefore, we have adjusted our test accordingly. This test is a kind of Unit test — the full test_app.py file:

import requests

# Define the base URL globally to avoid repetition in each test
BASE_URL = 'http://127.0.0.1:5000/'

def test_homepage():

    # Send a GET request to the homepage
    response = requests.get(BASE_URL)

    # Assert that the request was successful (status code 200)
    assert response.status_code == 200


def test_homepage_title():

    response = requests.get(BASE_URL)

    # For example, if the homepage includes a specific title or text
    assert b'<h1>Just Chill and enjoy some memes!</h1>' in response.content
    assert b'Infinite Memes' in response.content

def test_homepage_img_meme():

    response = requests.get(BASE_URL)

    # Check if the <img> tag is present in the response content
    img_tag_start = response.text.find('<img src="')
    assert img_tag_start != -1, "No <img> tag found on the page"

    # Extract the URL from the <img> tag's src attribute
    img_tag_end = response.text.find('"', img_tag_start + len('<img src="'))
    meme_pic = response.text[img_tag_start + len('<img src="'):img_tag_end]

    # Assert that meme_pic is not empty
    assert meme_pic != '', "meme_pic should not be empty"

We have received confirmation that our test has been successfully executed! Pytest marks a successfully executed test with a green dot, and a failed test with a red letter F.

Note: If you are debugging tests through the console and want to output the response, you can use the command $ pytest -s test_*.py to write to stdout. By using the s option, you will be able to output console messages inside the tests and debug them while they are running.

When new features are added to the application, the number of tests should be increased to ensure everything works correctly. Pytest allows you to run multiple tests from one file, grouping them together. It also offers markers that allow you to assign attributes and characteristics to test functions.

Web App Testing with Playwright, Pytest Alternative

Choosing the appropriate testing framework is crucial as it ensures an efficient and seamless verification process, helping to identify and fix issues early in the development stage.

Check out the comparison table we prepared for you ⬇️

Summary:

Pytest is an optimal solution for Python testing when you need a flexible and concise approach to writing tests. This tool is renowned for its versatility and supports various types of testing, including unit, functional, and API testing, as we can see just on top. It is especially helpful for writing clean, maintainable, and highly organised test code.

Playwright is a powerful tool for conducting E2E tests and browser automation, which can be very useful when testing Flask applications that have user interfaces, such as web pages; anyway, Playwright is also quite popular for API testing. Use Playwright when you need to simulate real user interactions on a web application in a browser.

Here are a few cases when Playwright should be used for testing Flask:

✅ Testing user interaction.
✅ Testing across different browsers.
✅ Testing applications that heavily use JavaScript.
✅ Simulating real-life scenarios.
✅ Testing API with an interface.
✅ Full application testing.
✅ Testing interface reactions and error handling.

At the same time, Playwright is not suitable for unit testing. For checking isolated components or functions in Flask, it’s better to use clear Pytest, as well as for testing APIs without interface interaction.

For maximum effectiveness, use Playwright for browser automation and Pytest to manage the test execution, structure, and reporting.

Setting up Playwright for Flask

Prerequisites

First, let’s install Playwright. Playwright recommends using the official Playwright Pytest plugin to write E2E and API tests. This command will install Pytest-Playwright along with the Playwright command-line tool:

pip install pytest-playwright

However, it uses the synchronous version of Playwright. If you need to work with the asynchronous version, you can install Playwright without the Pytest plugin using the following command:

pip install playwright

The next important step is downloading the browsers required for Playwright automation to work. Run the command:

playwright install

Now, move on to the next stage – creating the first test.

Writing Your First Tests with Playwright

First, let’s create a file called first_playwright_test.py and add our first test scenario with the following code to it:

You can run the Playwright first test for our Flask example application by following the command:

pytest first_playwright_test.py

This test is quite simple, but it already allows us to see how Playwright works. For example, we open our URL, but the test expects <title> Infinite meme, we got the error, as in truth it is Infinite Memes. If rewrite our testing framework in Playwright it looks like below:

import re
import requests
from playwright.sync_api import Page, expect

BASE_URL = "http://127.0.0.1:5000/"

def test_homepage_title_contains_infinite_memes(page: Page):
    page.goto(BASE_URL)
    expect(page).to_have_title(re.compile("Infinite Memes", re.IGNORECASE))

def test_homepage_has_main_heading(page: Page):
    page.goto(BASE_URL)
    expect(page.get_by_role("heading", name="Just Chill and enjoy some memes!")).to_be_visible()

def test_homepage_has_image_displayed(page: Page):
    page.goto(BASE_URL)
    meme_img = page.locator("img")
    expect(meme_img).to_be_visible()
    assert meme_img.get_attribute("src") != "", "Expected meme image src not to be empty"

def test_homepage_refresh_notice(page: Page):
    page.goto(BASE_URL)
    expect(page.locator("text=auto-refresh in 15 seconds")).to_be_visible()

# ✅ API test: verify the homepage returns HTTP 200 OK
def test_api_homepage_status():
    response = requests.get(BASE_URL)
    assert response.status_code == 200, f"Expected status 200, got {response.status_code}"

Running Playwright test for Flask Application with Codegen

Playwright has an automatic code generator — Codegen. This command opens a browser window and Playwright inspector for the specified URL, allowing you to track actions.

playwright codegen http://127.0.0.1:5000/

We will use Codegen Playwright’s graphical interface to check our simple Flask web application and gradually add more tests by performing interactions on the page.

After the page loads, you can navigate across the page, and all actions with conditions will be recorded automatically. Just select the Record. The inspector window shows how the code reacts to our actions on the page. We can then experiment with the behavior we are interested in.

Codegen attempts to create reliable text-based selectors. Some results might not be optimal, for example, too generic. Thus, we made some fixes in the generated code:

In our case, we want to precisely target elements by their existing on the page, by text. This example ensures that even minor changes in the interface will be detected during testing.

Although, as you know, there are unit tests to verify this at the code level, the ability to simulate a user action — registering and uploading a file — allows detecting issues at all levels of the application.

Importance of Presenting Test Results, Reporting

Real-time reporting helps track testing efforts. It detects defects. Gives insights into which areas are okay and which need more attention. This is obvious for release planning, especially within Agile teams. When stakeholders see transparent, detailed, and accurate test results, they can improve their testing strategies and processes. In turn, it increases their confidence in the software’s quality.

Test management system testomat.io synchronises automated and manual tests for both Platwright and Pytest frameworks in one place to see the common picture in informative reports and analytics.

In addition, find Flask’s built-in capabilities useful:

Reliable error handling is an important aspect of Flask application development. In Flask, exceptions can be effectively managed using try-except blocks, preventing crashes and providing users with understandable error messages. For instance, if an API request fails or invalid data is entered, exception handling will help form a helpful response instead of causing a crash. This improves the user experience and simplifies problem diagnosis.

For further analysis and troubleshooting, logging is advisable. In Flask, logging libraries can be used to record errors, making it significantly easier to track faults by their ID. Custom error pages can also be configured to display clear messages, such as for 404 errors or internal server failures.

Another useful tool is Flask’s built-in debugger, which provides detailed error traces in development mode, helping to quickly pinpoint their causes. It is essential to remember that in a production environment, debugging mode should be turned off to prevent sensitive information leaks. Using these methods enhances the stability of Flask applications and makes them more user-friendly.

Implementing logging in Flask applications is crucial for effective debugging and performance analysis. By using Python’s built-in logging module, various log levels can be captured, from debugging and informational messages to warnings and critical errors. This allows you to record events in the application, which helps quickly identify causes of failures and track user behavior.

To set up logging in Flask, the log level and record format should be defined early in the application’s setup. Logs can be stored in files for long-term analysis or output to the console for real-time monitoring during development. It is important to include additional contextual information in logs, such as timestamps and user actions. Well-integrated logging simplifies problem detection and improves application stability, ensuring its reliable operation.

Conclusion

Mastering testing and debugging techniques is key for every developer working with Flask applications. Implementing effective testing approaches, creating an appropriate environment, and using reliable diagnostic methods can significantly improve the stability and performance of products. It is important to remember that a thoroughly tested application not only ensures a better user experience but also simplifies its ongoing maintenance and development.

Playwright is a powerful tool for reliable E2E testing, and if you are working with Python, it is definitely worth considering. By integrating Playwright with Flask testing, you ensure the seamless operation of both the backend and frontend of your application, providing users with a reliable and user-friendly experience.